Skip to content

Adopt appendcol, appendpipe, multisearch to spath #5075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ykmr1224 merged 1 commit into from
Jan 27, 2026
Merged

Adopt appendcol, appendpipe, multisearch to spath #5075

ykmr1224 merged 1 commit into from
Jan 27, 2026

Conversation

@ykmr1224
Copy link
Collaborator

@ykmr1224 ykmr1224 commented Jan 27, 2026

Description

  • Adopt appendcol, appendpipe, multisearch to spath
    • Fixed each command to adjust fields when used with spath command (dynamic fields)

Related Issues

#4984

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 27, 2026 •
edited
Loading

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • The spath command now works with appendcol and multisearch commands (previously unsupported).

  • Bug Fixes

    • Improved dynamic field handling in query processing to ensure consistent field adjustment across multiple inputs.

  • Documentation

    • Updated spath command documentation to reflect expanded command compatibility.

✏️ Tip: You can customize this high-level summary in your review settings.

Walkthrough

This PR modifies field resolution and dynamic field handling to allow spath-related commands (appendcol, appendpipe, multisearch) to work with dynamic fields. The changes replace exception-throwing behavior with verification delegation to subqueries/subsearches, add bulk input alignment logic for dynamic fields, and extend test coverage. Documentation is updated to reflect that appendcol and multisearch are no longer restricted with spath.

Changes

Cohort / File(s) Summary
Field Resolution Visitor Dispatch
core/src/main/java/org/opensearch/sql/ast/analysis/FieldResolutionVisitor.java		 Replaces exception-throwing for appendcol, appendpipe, and multisearch with verification-and-traversal pattern. New method visitAppendPipe added; visitMultisearch removed. Commands now delegate requirement checks to subsearches before continuing traversal instead of aborting.
Dynamic Field Input Alignment
core/src/main/java/org/opensearch/sql/calcite/DynamicFieldsHelper.java		 Adds three new methods: adjustInputsForDynamicFields (aligns multiple inputs together), getRequiredStaticFields (collects static fields across inputs with dynamic fields), and toSortedList (utility). Refactors join input handling to use bulk adjustment. Updates adjustFieldsForDynamicFields to use MAP_APPEND when merging with existing dynamic fields.
Dynamic Field Integration
core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java		 Adds single call to adjustInputsForDynamicFields for multisearch subquery inputs before schema unification and union.
Documentation & Tests
docs/user/ppl/cmd/spath.md, integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLSpathCommandIT.java, ppl/src/test/java/org/opensearch/sql/ppl/parser/FieldResolutionVisitorTest.java		 Updates spath limitation docs to remove appendcol and multisearch restrictions. Adds 96 integration test lines for appendcol, appendpipe, multisearch scenarios with spath and dynamic fields. Adds 32 unit test lines covering field resolution for new append-style operators.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • Support spath with dynamic fields #5058: Directly related—both PRs modify the same visitor and helper classes (FieldResolutionVisitor, CalciteRelNodeVisitor, DynamicFieldsHelper) for dynamic field handling with spath commands.

Suggested reviewers

  • LantaoJin
  • ps48
  • kavithacm
  • joshuali925
  • anirudha
  • Swiddis
  • qianheng-aws
  • derek-ho
🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.52% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: adopting appendcol, appendpipe, and multisearch commands to work with spath.
Description check ✅ Passed The description directly addresses the changeset by explaining that appendcol, appendpipe, and multisearch are being adopted to work with spath, with dynamic field adjustments.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.40.5)
core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java

Comment @coderabbitai help to get the list of available commands and usage tips.

@ykmr1224 ykmr1224 added PPL Piped processing language calcite calcite migration releated enhancement New feature or request labels Jan 27, 2026
@ykmr1224 ykmr1224 marked this pull request as ready for review January 27, 2026 17:16
@ykmr1224 ykmr1224 merged commit 7630db8 into opensearch-project:main Jan 27, 2026
40 of 43 checks passed
asifabashar pushed a commit to asifabashar/sql that referenced this pull request Jan 27, 2026
@ykmr1224 @asifabashar
Adopt appendcol, appendpipe, multisearch to spath (opensearch-project…
4dd8f86 
…#5075)

Signed-off-by: Tomoyuki Morita <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@penghuo penghuo penghuo approved these changes

@Swiddis Swiddis Swiddis approved these changes

@ps48 ps48 Awaiting requested review from ps48 ps48 is a code owner

@kavithacm kavithacm Awaiting requested review from kavithacm kavithacm is a code owner

@derek-ho derek-ho Awaiting requested review from derek-ho derek-ho is a code owner

@joshuali925 joshuali925 Awaiting requested review from joshuali925 joshuali925 is a code owner

@dai-chen dai-chen Awaiting requested review from dai-chen dai-chen is a code owner

@YANG-DB YANG-DB Awaiting requested review from YANG-DB YANG-DB is a code owner

@mengweieric mengweieric Awaiting requested review from mengweieric mengweieric is a code owner

@vamsimanohar vamsimanohar Awaiting requested review from vamsimanohar vamsimanohar is a code owner

@seankao-az seankao-az Awaiting requested review from seankao-az seankao-az is a code owner

@MaxKsyunz MaxKsyunz Awaiting requested review from MaxKsyunz MaxKsyunz is a code owner

@Yury-Fridlyand Yury-Fridlyand Awaiting requested review from Yury-Fridlyand Yury-Fridlyand is a code owner

@anirudha anirudha Awaiting requested review from anirudha anirudha is a code owner

@forestmvey forestmvey Awaiting requested review from forestmvey forestmvey is a code owner

@acarbonetto acarbonetto Awaiting requested review from acarbonetto acarbonetto is a code owner

@GumpacG GumpacG Awaiting requested review from GumpacG GumpacG is a code owner

@LantaoJin LantaoJin Awaiting requested review from LantaoJin LantaoJin is a code owner

@noCharger noCharger Awaiting requested review from noCharger noCharger is a code owner

@qianheng-aws qianheng-aws Awaiting requested review from qianheng-aws qianheng-aws is a code owner

@yuancu yuancu Awaiting requested review from yuancu yuancu is a code owner

@RyanL1997 RyanL1997 Awaiting requested review from RyanL1997 RyanL1997 is a code owner

Assignees

No one assigned

Labels

calcite calcite migration releated enhancement New feature or request PPL Piped processing language

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ykmr1224 @penghuo @Swiddis